Hyperspectral image sensor with calibration

ABSTRACT

A method for calibrating an image sensor begins by illuminating a portion of the image sensor with an input light spectrum, where the input light spectrum includes light of known wavelength and intensity. The method continues by sampling an output for each optical sensor of the image sensor, where each optical sensor is associated with one or more optical filters and where each optical filter being associated with a group of optical filters of a plurality of groups of optical filters. Each optical filter of a group of optical filters is configured to pass light in a different wavelength range and at least some optical filters in different groups of the plurality of groups of optical filters are configured to pass light in substantially a same wavelength range. The method then continues by comparing a sampled output for each optical sensor of the plurality of optical sensors with an expected output and generating a calibration factor for each of at least a subset of the plurality of optical sensors and storing the generated calibration factors in memory.

CROSS REFERENCE TO RELATED APPLICATIONS

The present U.S. Utility Patent Application claims priority pursuant to35 U.S.C. § 120 as a continuation of U.S. Utility application Ser. No.16/814,234, entitled “INTEGRATED CIRCUIT FOR SPECTRAL IMAGING SYSTEM”,filed Mar. 10, 2020, which is a continuation of U.S. Utility applicationSer. No. 16/359,911, entitled “INTEGRATED CIRCUIT FOR SPECTRAL IMAGINGSYSTEM”, filed Mar. 20, 2019, issued as U.S. Pat. No. 10,620,049 on Apr.14, 2020, which is a continuation of U.S. Utility application Ser. No.15/059,715, entitled “INTEGRATED CIRCUIT FOR SPECTRAL IMAGING SYSTEM”,filed Mar. 3, 2016, issued as U.S. Pat. No. 10,260,945 on Apr. 16, 2019,which is a divisional of U.S. Utility application Ser. No. 13/482,860,entitled “INTEGRATED CIRCUIT FOR SPECTRAL IMAGING SYSTEM”, filed May 29,2012, issued as U.S. Pat. No. 9,304,039 on Apr. 5, 2016, which is acontinuation of PCT Application No. PCT/EP2010/068575, entitled“INTEGRATED CIRCUIT FOR SPECTRAL IMAGING SYSTEM”, filed Nov. 30, 2010,which claims priority pursuant to 35 U.S.C. § 119(e) to U.S. ProvisionalApplication No. 61/265,231, entitled “HYPERSPECTRAL CAMERA SYSTEM”,filed Nov. 30, 2009, all of which are hereby incorporated herein byreference in their entirety and made part of the present U.S. UtilityPatent Application for all purposes.

BACKGROUND OF THE INVENTION Field of the Invention

The disclosed technology relates to integrated circuits for an imagingsystem which has an array of optical sensors and an array of opticalfilters, and to corresponding systems and methods, and computerprograms, and more particularly to hyperspectral imaging (HSI) systems.

Description of the Related Technology Operation of Known HyperspectralImaging Systems

Hyperspectral imaging refers to the imaging technique of collecting andprocessing information from across the electromagnetic spectrum. Whereasthe human eye only can see visible light, a hyperspectral imaging systemcan see visible light as well as from the ultraviolet to infrared.Hyperspectral sensors thus look at objects using a larger portion of theelectromagnetic spectrum, as has been described at:http://en.wikipedia.org/wiki/Hyperspectral_imaging.

Certain objects leave unique ‘fingerprints’ across this portion of theelectromagnetic spectrum. These ‘fingerprints’ are known as spectralsignatures and enable identification of the materials that make up ascanned object. The hyperspectral capabilities of such imaging systemenable to recognize different types of objects, all of which may appearas the same color to the human eye.

Whereas multispectral imaging deals with several images at discrete andsomewhat narrow bands, hyperspectral imaging deals with imaging narrowspectral bands over a contiguous spectral range. It can produce thespectra for all pixels in the scene. While a sensor with 20 discretebands covering the VIS, NIR, SWIR, MWIR, and LWIR would be consideredmultispectral, another sensor with also 20 bands would be consideredhyperspectral when it covers the range from 500 to 700 nm with 20 10-nmwide bands.

Hyperspectral sensors collect information as a set of ‘images’. Eachimage represents a range of the electromagnetic spectrum and is alsoknown as a spectral band. These images each have two spatial dimensionsand if images of a series of different spectral bands are effectivelystacked to form a cube, then the third dimension can be a spectraldimension. Such a three dimensional hyperspectral cube is a usefulrepresentation for further image processing and analysis. The precisionof these sensors is typically measured in spectral resolution, which isthe width of each band of the spectrum that is captured. If the scannerpicks up on a large number of fairly narrow frequency bands, it ispossible to identify objects even if the objects are only captured in ahandful of pixels. However, spatial resolution is a factor in additionto spectral resolution. If the pixels are too large, then multipleobjects are captured in the same pixel and become difficult to identify.If the pixels are too small, then the energy captured by eachsensor-cell is low, and the decreased signal-to-noise ratio reduces thereliability of measured features.

Current hyperspectral cameras produce a hyperspectral datacube or imagecube, consisting of a stack of 2D images in the x-y plane of the scenein which each image of the stack contains information from a differentfrequency or spectral band. The spectral range that is captured is notlimited to visual light, but can also span infra red (IR) and/or ultraviolet (UV). The 3D Image Cube is captured by a hyperspectral imager,using a sensor that is inherently a 2D sensor. Therefore some form ofscanning needs to be used, as is shown in FIG. 1 which shows aperspective representation of a cube with the spectral dimensionextending vertically, and four views a) to d) of slices of the cube asfollows:

Topview (a) shows the scene that needs to be captured. Left sideview (b)shows a vertical slice from the cube, representing an image obtained bya line scanner: all spectral bands are captured for one spatial line ofthe scene resulting in a 1D view. Line scanners or pushbroom systemsthus capture a single line of the 2D scene in all spectral bands inparallel. To cover all spatial pixels of the scene, this type of systemthen scans different lines over time, for example by relative movementof the scanner and the scene.

Right sideview (c) shows a horizontal slice showing an image obtained bya starer: the complete 2D scene is captured in one spectral band.Starers or staring systems capture the complete scene in a singlespectral band at a time with a 2D array of sensors and scan overdifferent spectral bands in order to produce the 3D hyperspectral imagecube. Bottom view (d) shows a sloping or diagonal slice through thecube, representing an image obtained by a hybrid line scanner/starer:the complete 2D scene is captured, but every spatial line is at adifferent height of the cube and so is a different spectral band. Inthis case a complete spatial image is acquired, but with every line at adifferent spectral band. In a single frame different spectral bands arethen captured for different spatial lines. To capture the complete 3Dimage cube, with all spectral bands for all spatial lines, a combinedspatial/spectral scanning is still needed, for example by relativemotion between the scene and the 2D sensor array.

Construction of Known Hyperspectral Imaging Systems

Hyperspectral imaging systems or cameras can consist of differentdiscrete components, e.g. the optical sub-system for receiving theincoming electromagnetic spectrum, the spectral unit for creating thedifferent bands within the received spectrum and the image sensor arrayfor detecting the different bands. The optical sub-system can consist ofa single or a combination of different lenses, apertures and/or slits.The spectral unit can consist of one or more prisms, gratings, opticalfilters, acousto-optical tunable filters, liquid crystal tunable filtersetc or a combination of these.

A primary advantage of hyperspectral imaging is that, because an entirespectrum is acquired at each point, the operator needs no priorknowledge of the sample, and post-processing allows all availableinformation from the dataset to be mined. The primary disadvantages arecost and complexity. Fast computers, sensitive detectors, and large datastorage capacities are needed for analyzing hyperspectral data.Significant data storage capacity is necessary since hyperspectral cubesare large multi-dimensional datasets, potentially exceeding hundreds ofmegabytes. All of these factors greatly increase the cost of acquiringand processing hyperspectral data.

State-of-the-art hyperspectral imagers are therefore either researchinstruments as they are too slow and too expensive or either designedfor a dedicated industrial application thereby lacking flexibility.

SUMMARY OF CERTAIN INVENTIVE ASPECTS

A first aspect provides an integrated circuit for an imaging system. Aneffect of these features is that read out from the array of opticalsensors can be speeded up or that a larger array can be used for a givenspeed of read out. This faster readout can reduce blur caused byrelative movement of the array of sensors and the subject being imaged,or can increase a resolution or quality of the image. The groups ofsensors can be arranged in various ways, such as interleaved, orlinearly concatenated for example. Image artifacts arising from thepattern of the groups can be compensated by subsequent image processingif necessary.

A second aspect provides an integrated circuit for an imaging system. Aneffect of having the thickness of the optical filter vary so as toincrease at some points and decrease at other points along the line isthat it enables neighboring optical filters to be either both thicker orboth thinner, to create ridges or valleys, or to enable clusters ofoptical filters to cover overlapping spectral bands for example.

A third aspect provides an integrated circuit for an imaging system. Aneffect of such variation of thickness along the strip is to improvespectral precision in the sensing or improve yield, or reduce a need forimage processing, or enable larger arrays for a given yield orprecision.

A fourth aspect provides an integrated circuit for an imaging system. Aneffect of the read out circuitry having a wavelength selector forselecting between or interpolating between read out signals ofcorresponding pixels of different optical filters is that it enables forexample spectral subsampling or spectral shifting, to compensate forvarious possible distortions. This in turn can enable yield increasesfor a wafer and/or cost decrease, since more variation in thickness canbe tolerated for a given accuracy in wavelength passed and thereforedetected. Another aspect provides an imaging system having such anintegrated circuit. Other aspects provide corresponding methods ofimaging using such systems, and corresponding computer programs forimage processing of a spectral cube.

Any of the additional features can be combined together and combinedwith any of the aspects. Other advantages will be apparent to thoseskilled in the art, especially over other prior art. Numerous variationsand modifications can be made without departing from the claims of thepresent invention. Therefore, it should be clearly understood that theform of the present invention is illustrative only and is not intendedto limit the scope of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS

How the present invention may be put into effect will now be describedby way of example with reference to the appended drawings, in which:

FIG. 1 shows the Hyperspectral Image Cube Acquisition.

FIG. 2 shows an optical filter using Fabry-Pérot wavelength selection.(a) Fabry-Pérot working principle, with multiple light rays beingreflected, which results in constructive and destructive interference,based on the wavelength of the light, on the distance 1 between thesemi-mirrors and the incident angle θ. (b) Higher orders are alsoselected, which results in an order selection problem.

FIG. 3 shows the definition of optical parameters of a filter.

FIG. 4 shows the dependence of angle of incidence onto wedge on the sizeof the exit pupil.

FIG. 5 shows the sensitivity of an optical filter in the form of aFabry-Perot interferometer to the incident angle.

FIGS. 6 a-b show an integrated imaging system (a) cross-section (b)Topview.

FIG. 7 a illustrates the variation of selected wavelengths as a functionof cavity length for a Fabry-Perot interferometer, and FIGS. 7 b-7 eillustrate transmission spectra at specific cavity lengths annotated inFIG. 7 a , according to embodiments.

FIG. 8 shows the principle of binary or logarithmic patterning of astep-like structure.

FIGS. 9 -a to e shows a schematic process flow for manufacturing aFabry-Perot interferometer.

FIGS. 10 -a to e shows an alternative schematic process flow formanufacturing a Fabry-Perot interferometer.

FIG. 11 shows the effect of processing tolerances on the filtercharacteristics.

FIG. 12 shows an integrated circuit in the form of an integrated imagingsystem designed to tolerate processing technology tolerances.

FIG. 13 shows an integrated imaging system having overlap of severalbands taking care of etching tolerances.

FIG. 14 shows an integrated imaging system whereby filters arere-ordered.

FIGS. 15 a-c shows the effect of the image sensor on the performance ofthe filter.

FIG. 16 shows the read-out of integrated imaging system.

FIG. 17 shows the read-out of integrated imaging system having more than1 line of sensors underneath a optical filter.

FIG. 18 shows an integrated imaging system combined with an objectivelens into a system.

FIG. 19 shows an integrated imaging system combined with a collimator.

FIG. 20 shows the effect of collimation on spectral resolution.

FIG. 21 shows the relation between slit size and spectral resolution ofan integrated imaging system.

FIGS. 22 a-b shows the effect of aperture size on spectral resolution ofintegrated imaging system.

FIGS. 23, 24, 27 and 28 show schematic views of integrated circuitsaccording to one embodiment.

FIGS. 25 and 26 show alternative profiles for the thicknesses of theoptical filters, having increases and decreases in thickness.

FIG. 29 shows a schematic view of an imaging system according to anembodiment.

FIG. 30 shows a side view of optical parts of an imaging system having acollimated system with a wedge filter array.

FIG. 31 shows a view of an uncollimated system with an integratedcircuit having a wedge filter array.

DETAILED DESCRIPTION OF CERTAIN ILLUSTRATIVE EMBODIMENTS

The present invention will be described with respect to particularembodiments and with reference to certain drawings but the invention isnot limited thereto but only by the claims. The drawings described areonly schematic and are non-limiting. In the drawings, the size of someof the elements may be exaggerated and not drawn on scale forillustrative purposes.

Where the term “comprising” is used in the present description andclaims, it does not exclude other elements or steps. Where an indefiniteor definite article is used when referring to a singular noun e.g. “a”or “an”, “the”, this includes a plural of that noun unless somethingelse is specifically stated.

The term “comprising”, used in the claims, should not be interpreted asbeing restricted to the means listed thereafter; it does not excludeother elements or steps.

Elements or parts of the described receivers may comprise logic encodedin media for performing any kind of information processing. Logic maycomprise software encoded in a disk or other computer-readable mediumand/or instructions encoded in an application specific integratedcircuit (ASIC), field programmable gate array (FPGA), or other processoror hardware.

References to software can encompass any type of programs in anylanguage executable directly or indirectly by a processor.

References to logic, hardware, processor or circuitry can encompass anykind of logic or analog circuitry, integrated to any degree, and notlimited to general purpose processors, digital signal processors, ASICs,FPGAs, discrete components or transistor logic gates and so on.

References to optical are intended to encompass at least wavelengthswithin the human visible wavelength range and also infra redwavelengths, and shorter wavelengths, extending into the ultra violetbands, where the sensitivity to manufacturing variations in thickness ofthe optical filter are even more pronounced. In some embodiments, theoptical filters and optical sensors can be limited to a range which isany subset of these wavelengths, for example visible wavelengths only,or visible and shorter wavelengths.

References to arrays of optical filters or arrays of optical sensors areintended to encompass 1-dimensional linear arrays, 2-dimensional arrays,rectangular or non rectangular arrays, irregularly spaced arrays, andnon planar arrays for example.

References to integrated circuits are intended to encompass at leastdies or packaged dies for example having the array of optical filtersmonolithically integrated onto the array of sensors, or devices in whichthe array of optical filters is manufactured separately and added lateronto the die or into the same integrated circuit package.

References to a spectrum of wavelengths are intended to encompass acontinuous spectrum or a range of nearly adjacent discrete bands forexample.

References to pixels being read out in parallel are intended toencompass instances in which all pixels have a separate line for readout, and instances where two or more pixels share a line and are outputenabled at different times, giving a partially parallel arrangement.

Furthermore, the terms first, second, third and the like in thedescription and in the claims, are used for distinguishing betweensimilar elements and not necessarily for describing a sequential orchronological order. It is to be understood that the terms so used areinterchangeable under appropriate circumstances and that the embodimentsof the invention described herein are capable of operation in othersequences than described or illustrated herein.

Moreover, the terms top, bottom, over, under and the like in thedescription and the claims are used for descriptive purposes and notnecessarily for describing relative positions. It is to be understoodthat the terms so used are interchangeable under appropriatecircumstances and that the embodiments of the invention described hereinare capable of operation in other orientations than described orillustrated herein.

Reference throughout this specification to “one embodiment” or “anembodiment” means that a particular feature, structure or characteristicdescribed in connection with the embodiment is included in at least oneembodiment of the present invention. Thus, appearances of the phrases“in one embodiment” or “in an embodiment” in various places throughoutthis specification are not necessarily all referring to the sameembodiment, but may. Furthermore, the particular features, structures orcharacteristics may be combined in any suitable manner, as would beapparent to one of ordinary skill in the art from this disclosure, inone or more embodiments.

Similarly it should be appreciated that in the description of exemplaryembodiments of the invention, various features of the invention aresometimes grouped together in a single embodiment, figure, ordescription thereof for the purpose of streamlining the disclosure andaiding in the understanding of one or more of the various inventiveaspects. This method of disclosure, however, is not to be interpreted asreflecting an intention that the claimed invention requires morefeatures than are expressly recited in each claim. Rather, as thefollowing claims reflect, inventive aspects lie in less than allfeatures of a single foregoing disclosed embodiment. Thus, the claimsfollowing the detailed description are hereby expressly incorporatedinto this detailed description, with each claim standing on its own as aseparate embodiment of this invention.

Furthermore, while some embodiments described herein include some butnot other features included in other embodiments, combinations offeatures of different embodiments are meant to be within the scope ofthe invention, and form different embodiments, as would be understood bythose in the art. For example, in the following claims, any of theclaimed embodiments can be used in any combination.

In the description provided herein, numerous specific details are setforth. However, it is understood that embodiments of the invention maybe practiced without these specific details. In other instances,well-known methods, structures and techniques have not been shown indetail in order not to obscure an understanding of this description.

The invention will now be described by a detailed description of severalembodiments of the invention. It is clear that other embodiments of theinvention can be configured according to the knowledge of personsskilled in the art without departing from the technical teaching of theinvention, the invention being limited only by the terms of the appendedclaims.

Introduction to Some Issues Addressed by the Embodiments:

It is desirable to have a combined spectral unit with image sensorarray. This integrated component needs to be combined with an opticalsub-system to form a complete hyperspectral camera system. Such ahyperspectral imaging system should be compact, be capable ofmanufacture at low cost, and be reconfigurable. In certain aspects,process technology aspects are combined with the system integration andimage processing techniques to alleviate the integrated circuitmanufacturing process requirements.

In some examples, a hyperspectral imaging system is disclosed comprisingan integrated circuit with a spectral unit monolithically integratedwith the array of optical sensors forming the image sensor array.

In a preferred embodiment the spectral unit is integrated with the imagesensor array using semiconductor process technology, i.e. the spectralunit is post processed on the substrate comprising the image sensorarray using semiconductor process technology and process steps. Examplesof such semiconductor technology arecomplementary-metal-oxide-semiconductor (CMOS) processing, whereby theimage sensor array is a CMOS sensor, and charge-coupled-device (CCD)processing, whereby the image sensor array is a CCD sensor. Thesemanufacturing techniques are ideally suited for producing integratedelectronic circuits. Such monolithic integration allows manufacturing atlow cost while offering a higher performance as no interface layers areneeded to attach the spectral unit to the substrate. Hence stray lighteffects are considerably reduced.

Given the large range of technology generations, one can choose tomanufacture the IMEC sensor in a lower cost technology having a largecritical dimension (CD), e.g., 130 nm, resulting a larger pixels andsmaller spatial resolution of the image sensor array. Alternatively onechoose to manufacture the image sensor array in a state in a higher costtechnology having a smaller critical dimension (CD), e.g., 45 nm,resulting a smaller pixels and higher spatial resolution of the imagesensor array.

The image sensor array can be a front-illuminated sensor, whereby thespectral unit is post processed on top of the substrate comprising thesensor. Optionally this substrate is thinned afterwards thereby removingthe bulk of the substrate and leaving a thin slice containing the imagesensor array and the spectral unit monolithically integrated therewith.Alternatively the image sensor array can be a back-illuminated sensor,whereby first the substrate comprising the sensor is thinned from thebackside onwards. On backside the thinned substrate the spectral unit isthen post processed.

Preferably the spectral unit is a sequential 1D or 2D array ofFabry-Pérot filters. This array can be monotonic whereby the thicknessof the Fabry-Pérot filters decreases in a monotonic way from one side ofthe array to the other. Alternatively this array can be non-monotonicwhereby the thickness of the Fabry-Pérot filters varies in anon-monotonic way from one side of the array to the other side. Methodsfor manufacturing such Fabry-Pérot filters are disclosed.

Although any order of Fabry-Pérot filters can be manufactured,preferably only 1st order Fabry-Pérot filters are formed on the imagesensor array thereby reducing the complexity for removing and/orblocking higher order components. Hence the complexity of operating thehyperspectral system is reduced. As the spectral unit is directly postprocessed on the substrate comprising the sensor, the spectral unit canbe made thin and such a 1st order Fabry-Pérot filter can bemanufactured. A monolithically integrated hyperspectral imaging with a1st order Fabry-Pérot filter as spectral unit typically doesn't requirea focusing lens in the optical subsystem.

Examples of complete hyperspectral imaging systems comprising theoptical subsystem and the monolithically integrated spectral unit andimage sensor array are disclosed. These complete imaging systems exploitfrom the benefits of the monolithically integration to allow freedom indesigning the optical subsystem.

Furthermore, methods for designing and operating a hyperspectral imagingsystem according to embodiments of the first aspect are also disclosed.These design and operating methods exploit the manufacturing features ofthese monolithically integrated imaging systems thereby tolerating alarger manufacturing window.

In some embodiments spectral oversampling is used to correct fordeficiencies and process tolerances in the manufacturing technology. Thehyperspectral imaging system is designed to have a higher spectralresolution and a higher number of bands than required by the targetedapplication(s). The thus designed imaging system has a reducedsensitivity of the Fabry-Perot filters to processing tolerancesintroduced, in particular by the tight specifications of a first orderFabry-Pérot filter. In addition, such a design enables a configurablereduction in spectral resolution by tuning the optical system at runtimefor gaining speed. The need for a collimator and slit is therebyeliminated resulting in a lower cost hyperspectral imaging system.

In some embodiments, range extension is used to correct for deficienciesand process tolerances in the manufacturing technology. The sequential1D or 2D array of Fabry-Pérot filters is designed in a particularnon-monotonous ordering, range extensions and intentionaloverlap/reproduction of steps. The thus designed imaging system has areduced sensitivity of the Fabry-Pérot filters to processing tolerancesintroduced, in particular by the tight specifications of a first orderFabry-Pérot filter. In addition the design of the filters, e.g. thethickness which defines the cavity length of the filters, can take intoaccount the location of the particular filter on the chip to reduce thedependency on variations in the incident angle of the incomingelectromagnetic spectrum.

Monolithic Integration:

The filter is post-processed on top of an image sensor array and everystep is aligned with a single or multiple rows or columns of the imagesensor array. Every step of the wedge filters out a different spectralband. As a result, the sensor and wedge filter combination can be usedin hyperspectral imagers of the pushbroom, line scanner type or thehybrid line scanner/starer type. A hyper spectral camera system cancomprise an optical filter post-processed on an image sensor array asdefined in the above, the system further comprising an objective lensand/or slit and/or a collimator.

The integrated spectral module is an integrated circuit forming asubsystem of this camera, and built from different optical line filtersintegrated on top of an image sensor. Existing wedge filters arediscrete components that are assembled onto the image sensor postproduction. As a result of the monolithic integration that is part ofone aspect of the disclosure, in which the filter is directly postprocessed on top of the imager, the amount of stray light between thefilter and the image sensor can be significantly reduced. As a resultthe spectral resolution is improved with respect to discretelyintegrated filters. Preferably semiconductor imagers such as CMOSimagers or CCD imagers are used to monolithically integrate theFabry-Perot filter.

The proposed hyperspectral module can be monolithically integrated,meaning that the filter structures are directly post-processed on top ofthe image sensor. This integration has very important advantages andsome consequences, compared to filter structures that are separatelyproduced and then assembled with the imager later. Advantages ofmonolithic integration are cost reduction through standard CMOSproduction steps, reduction in stray light, allow design for first orderand avoid the need for a focusing lens.

When compared to a hybrid integration, in which the filter structuresare separately produced and then assembled with the image sensor intothe hyperspectral module, the proposed approach has some very clearadvantages.

Firstly, the combination of both production sequences into one combinedflow leads to an overall simplification and cost reduction in theproduction, when compared to a hybrid integration of the filterstructures that are separately produced and then later assembled withthe sensor into the module. This is especially the case for this filter,as the post-production of the filter structures requires only CMOScompatible fabrication steps, like deposition, patterning and etching.By adding these steps to the normal production flow of the image sensor,expensive, error prone and labor intensive assembly steps are prevented.E.g., for a filter with 3 layers of oxide and amorphous silicon in theBragg stack and 127 steps in the cavity, around 50 lot-turns are needed,giving an additional cost of more or less 20% with respect to standardCMOS imagers. The number of lot turns for the deposition of the top andbottom mirror layers can even be reduced if the different layers can bedeposited, one after the other, in the same tool.

Secondly by manufacturing the filter structure directly on top of thepixels of the imager, photons can pass directly from the filter into thepixel below. In the case of front side illuminated sensors, photons willfirst pass through the metallization layers and some dielectric layers.When the filter structure is produced separately and stacked on top ofthe image sensor, there will always be a non-functional layer or gap inbetween both structures.

Even when the filter and substrate combination is flipped and the filteris located in between the supporting substrate and the image sensor, thelight will pass through the substrate first, then through the filter andfinally through a thin air or glue gap, before it hits the image sensorphotodiodes. When a filter structure is combined with an image sensor,be it stacked on top of each-other with air or glue between thedifferent layers, this extra substrate between the filter structure andthe underlying rows of pixels will always give rise to a certain amountof performance degradation because of:

Cross Talk

Photons that exit the filter structure above a certain pixel can crossthe gap and fall onto a neighboring pixel. This effect will be heavilyreduced when the gap is reduced or completely removed by a directpostprocessing of the filter onto the pixels. There can still be somecross-talk as a result of the thickness of the filter itself however, asa photon that enters the filter above one pixel can still propagatethrough the filter and fall onto a neighboring pixel. This is reduced bydesigning thinner filters and by controlling the angle of incidence.

Stray Light

The extra non-functional layer gives rise to extra reflections on itsboundaries if the refractive indices are not matched (See Equation 8below) and therefore to extra stray light on top of the cross-talkdiscussed above. By reducing the effective distance S between the filterand the pixel array of the image sensor for different incident anglesstray light is reduced. For a smaller distance S, e.g. 1 nm, thedistance that is traveled by the stray light (D) is well within normalpixel dimensions (e.g. 1 to 15 nm). This is not the case for moremacroscopic integration distances, e.g. 1 mm substrate, in which casethe distance of the traveled light D ranges over tens to hundreds ofpixels, leading to a severe deterioration of the spatial and spectralresolution. In some cases, the distance D can become so large, anadditional focus lens is required to focus the light back onto thepixel.

Parasitic Fabry-Perot Because on Top of the Stray Light:

Additionally, as indicated in the previous item, the dielectric stackand metals on top of the photodiodes reflect part of the light. Togetherwith the gap because of the heterogeneous integration and the bottommirror of the cavity, this forms a parasitic Fabry-Perot interferingwith the actual one. This process can be optimized with the monolithicintegration as the dielectric layers in the imager become part of thebottom Bragg stack, made in similar materials (e.g. oxide) and which isnot very sensitive to the width of these layers.

One important reason why the hybrid filter structures that arepost-production assembled onto the image sensors suffer heavily fromthis problem, is the fact that the construction of very thin filterstructures separately, requires the additional insertion of a(transparent) support structure to mechanically support the filters andenable the stacking. When this layer is placed between the filter andthe image sensor, the non-functional gap consists of this layer and anadditional air or glue gap in between the support layer and the imagesensor. When the support structure is placed on top, it can alsogenerate additional reflections and should be optimized separately (e.g.by adding anti-reflective coatings), but still there will be an air orglue layer in between the filter and the image sensor. All of this canbe made redundant by post-processing the filter structures directly ontop of the image sensor, as has been discussed above.

Thirdly, the monolithic integration, combined with very precise CMOSfabrication techniques, enables the construction of filter structureswith much smaller thicknesses. As discussed later, the Fabry-Perotfilter structure is designed to select a certain wavelength by tuningthe cavity length. Thinner filters are less sensitive to the incidentangle, as the internal reflections in the filters cover less distancefor non-perpendicular incidence. A thicker filter will suffer from alarger displacement D of the transmitted beams, ranging well over 10 mm.This leads to a severe reduction in spatial and spectral resolution, asthe light that passes through the filters will fall onto other rows orcolumns of pixels. This macroscopic filter hence requires a focusinglens. The thin filters are much less sensitive to this and thedisplacement D stays in most cases below the pixel dimensions, i.e.preferably in the 1 to 10 nm range, for all but the largest angles ofincidence and the smallest pixels sizes. Traditional productiontechniques, in combination with hybrid integration of the filterstructure and the image sensor, can not reach the required accuracy tofabricate Fabry-Perot filters of the first order. Hence, higher orderFabry-Perot structures have to be used. In that case, additionaldichroic or other filters have to be added to the module, in order toselect the required order only. This gives rise to additional energyloss, additional costs and hence reduced overall system optimality.

Finally, when a Fabry-Perot filter is placed some distance away from theimage sensor, the output of the filter exhibits phase differences that,when focused by a lens, take on the form of concentric circles. Theconcentric circles are a result of the different interfering waves whereyou have at different locations constructive and destructiveinterference. The focusing lens is needed for macroscopic filtersbecause of the large distances covered by reflections inside the filterand in order to focus all these reflections back onto one pixel. In thedisclosed integrated imaging module, the distance between the filterstructure and the image sensor is very small and as the filter isdesigned for the first order, there is no need for a focusing lens. Thinfilters don't need this focusing lens, because internal reflectionscover much smaller distances and in the case of the proposed filter, alllight still falls in one pixel (after a very large number of internalreflections, the energy that is left in the light ray that exceeds thesize of a single pixels is negligible). The concentric circles that arethe result of the phase difference, will still be there, but will all befocused inside the same pixel and their effect is all integrated in theoutput of that pixel.

The direct post-processing of the filter structure on top of an activeIC, in this case the image sensor, should be compatible with thecontamination, mechanical, temperature and other limitations of that IC.This means that e.g. none of the steps used in the fabrication of thefilter can use materials or processing steps that would damage the imagesensor below.

As will be discussed below, one of the most important limitations is therestriction on the available materials, taking into account the CMOSproduction environment. In the proposed filter, the material selectionhas been done such that standard materials have been used, that arefully compatible with standard processing. Using some materials is notpossible, e.g. Au or Ag, as they tend to diffuse into the differentlayers and into the tools and thereby negatively affect the yield of thecurrent and even future processing steps. In some cases, such a layercan still be acceptable as a final step (top layer), when the depositionis done outside of the normal processing line and when the tool is onlyused for that purpose. This can only be done as a final step, as thewafer can not enter the normal flow after this operation. Anotherlimitation, related to the material selection, is the temperature budgetor the temperature window that is still available for processing. Inorder to perform the post-processing without damaging the image sensor.To prevent damage, the maximal temperature of the processing stepsshould not exceed a certain maximum, e.g. 400 degrees C. This alsorestricts the choice of materials and crystallization that is availablefor the design. With respect to a hybrid approach, where the imagesensor and a separately produced filter structure are assembled into amodule later, there is less freedom here. In case of a monolithicdesign, the restrictions have to be taken into account throughout thedesign. If certain design choices can be made during the design of theimage sensor itself, to relax the constraints on the processing of thefilter (e.g. to raise the allowed temperature for post-processing), thiscan be taken into account too. This then leads to an optimizationproblem at the module level, instead of for the image sensor and thefilter structures separately. The restriction on the filter structuresalways apply, as it is processed later and on top of the image sensor.

Optical Filter

Every pixel of the image sensor can have its own optical filter,sensitive to one specific wavelength. The organization of differentoptical filters on the sensor depends on its usage. A line scannerrequires the same wavelength selectivity for every pixel on the sameline, in which case it is here referred to as a line filter. Differenttypes of filters exist. In one embodiment, the type that is used is theFabry-Perot interferometer.

Fabry-Perot Filter:

FIG. 2 shows Fabry-Perot wavelength selection. (a) Fabry-Pérot workingprinciple, with multiple light rays being reflected, which results inconstructive and destructive interference, based on the wavelength ofthe light, on the distance 1 between the semi-mirrors and the incidentangle θ. (b) Higher orders are also selected, which results in an orderselection problem. The filter operation is based on the well-knownFabry-Pérot principle, in which the height of each step is tuned to thefiltered spectral band. Each step forms a resonant cavity of which theresonance frequency is determined by the height of the step. On the topand bottom of the cavity, a semi-transparent mirror is placed topartially reflect the light ray. Because of the reflections, an opticalpath difference is introduced resulting in destructive and constructiveinterference (depending on the incoming wavelength), as shown in FIG. 2a.

The Fabry-Perot Filter is made of a transparent layer (called cavity)with two reflecting surfaces at each side of that layer. Thetransparency and reflectivity of the surfaces have to be considered withrespect to the wavelength range that the Fabry-Perot filter is targetedat. The transmission of the light as a function of the wavelengths showsa narrow peak around a central wavelength corresponding to the resonancein the cavity. As indicated in FIG. 2 a , light in the cavity isreflected multiple times, introducing a path length difference and aphase shift for light passing through the filter. The multiple lightrays at the output cause interference depending on the phase shiftintroduced in the cavity. The many interfering light rays lead to a veryselective optical filter for which the transmission function is given byEquation 1.

$\begin{matrix}{T_{e} = \frac{T^{2}}{1 + R^{2} - {2R\cos\delta}}} & (1)\end{matrix}$with δ the introduced phase shift (for an incident angle θ) equal to:

$\begin{matrix}{\delta = {4\frac{\pi}{\lambda}\cos\theta}} & (2)\end{matrix}$

Constructive interference occurs when this phase shift is equal to zeroor a multiple of 2. In that case, the numerator and denominator ofequation 1 are equal and the transmission is 100%. Equation 3 describesthe transmission of the Fabry-Perot filter as a function of the length,angle of incidence and refractive index of the cavity. From thisequation, a Fabry-Perot filter can be designed for a certain wavelength,by varying the cavity length. In case of constructive interferenceoccurs the numerator and denominator of Equation 1 are equal and thetransmission is 100%. Equation 3 gives the relation between thewavelength for which the transmission is 100% as a function of thelength, angle of incidence and refractive index of the cavity. From thisequation, a Fabry-Perot filter can be designed for a certain wavelength,by varying the cavity length 1.mλ=nl cos θ  (3)

The central wavelength of the Fabry-Perot filter is only one ofimportant optical parameters. As constructive interference alwayshappens when the phase shift is equal to a multiple of 2, multiplewavelengths, called higher orders, will pass the filter. As indicated inFIG. 3 , the wavelength separation between two transmission peaks of thefilter is called the Free Spectral Range. The larger this parameter,less problems with higher orders wavelengths will occur. A Fabry-Perotinterferometer designed for first order wavelength will provide amaximum Free Spectral Range. Indeed, for a central wavelength of 700 nmin first order, the Free Spectral Range is 350 nm to the second order at350 nm. If the central wavelength in first order is 1400 nm, then 700 nmis selected in second order and the third order is 466 nm, which reducesthe Free Spectral range to 233 nm for 700 nm. A second parameter is thequality of the filter, which is defined as the bandwidth of the filterrelative to the central wavelength. The bandwidth is expressed ad theFull Width Half Maximum or FWHM of the filter, which is defined as thewidth of the passband at half the maximum transmission, as shown in FIG.3 . A third parameter, also indicative of the quality of the opticalfilter, is the finesse F of the Fabry Perot interferometer defined inEquation 4 as the relation between the Free Spectral Range Δλ and theFWHM δλ. For a fixed Free Spectral Range, a higher finesse automaticallyleads to a lower FWHM or better spectral resolution (see below). Asshown in Equation 5, the finesse only depends on the reflectivity of thereflecting surfaces. The higher this reflectivity, the higher thefinesse and the narrower the bandwidth or FWHM of the optical filter forthe same Free Spectral Range will be.

$\begin{matrix}{F = \frac{\Delta\lambda}{\delta\lambda}} & (4)\end{matrix}$ $\begin{matrix}{F = \frac{\pi \cdot \sqrt{R}}{1 - R}} & (5)\end{matrix}$

FIG. 4 shows two Fabry-Perot interferometers with equal cavity lengthbut with different reflecting surfaces. The angle of incidence was 0°and the cavity was filled with air with a refractive index equal to 1.This leads to two different filters with different Full Width HalfMaximum for the same Free Spectral Range. The cavity of both filters is450 nm resulting in a central wavelength of 900 nm and a second orderwavelength at 450 nm. The two different implementations haverespectively a low (2) and high (1) finesse resulting in low (2) andhigh (1) FWHM for the same Free Spectral Range.

A fourth parameter for the Fabry-Perot filter is the spectralresolution, i.e. the minimal difference in central wavelength of twoneighboring spectral bands that can be resolved. This parameter dependson both the position of the central wavelength and the bandwidth of thefilter. Two neighboring spectral filtered bands are the to bedistinguishable if the peak in their transmission characteristic crossesat half the maximum (the 3 dB point) or below, i.e. at or below thelocation where the FWHM is calculated. When relating the location of asingle filter to the sampling of a complete wavelength range, it isassumed that the length of the cavity can be controlled very preciselyduring processing and that one is able to position next spectral band sothat its 3 dB point actually crosses in that 3 dB point. If the spectralrange of interest is then sampled with a range of line filters, eachpositioned such that their pass band crosses at the 3 dB point, thespectral resolution of the hyperspectral module is equal to the FWHM ofthe optical filters, under the assumption that the FWHM of twoneighboring filters is the same.

As indicated in equation 3 and illustrated in FIG. 5 the centralwavelength of the interferometer depends on the angle of incidence ofthe light. For a Fabry-Perot filter, this dependence is a cosinerelationship, which is not very sensitive at angles around 0°, i.e.light perpendicular to the surface of the optical filter. This is incontrast to gratings, for which the dependency of the wavelengthselection follows a sine relationship, which is much more sensitive tovariations around 0° degrees. A Fabry-Perot interferometer can tolerateslight variations on the angle of incidence. This feature can be used atthe systems level to improve on speed, sensitivity, etc.

Design of the Optical Filter

The design and performance of the reflecting surfaces on both sides ofthe cavity are crucial to the performance of a Fabry Perot opticalfilter. A Fabry-Perot optical filter with high finesse, and thus goodspectral resolution, can only be obtained by using highly reflectivemirrors. A second important parameter of the mirrors is theirabsorption, as this will determine the efficiency of the filter. If afull range of Fabry-Perot optical filters has to be constructed over acertain wavelength range, it is beneficial that these two parameters(reflectivity and absorption) stay as constant as possible over thisspectral range. In that case, the wavelength range can becovered/sampled by varying only the cavity length of the Fabry-Perotfilters and the materials and mirror layers can be kept constant. Theselected wavelength range has to match the sensitivity of the selectedimage sensor, which is the second component of the module

Current solutions proposing monolithic Integration use specificnon-standard sensor designs, increasing the cost or decreasing thespeed. Switching to CMOS compatible processing steps on CMOS sensorsraises integration issues as it has consequences on e.g. the materialselection, due to contamination and temperature budget. Metals like Agfor the bottom mirror can't be used. State of the art Fabry-Perotfilters needs to use Al, causing a serious decrease of the filterquality or optical throughput (speed). Dielectric stacks are preferredbut the contamination level and temperature budget limits the materialselection. Process compliant materials needed having the correctcombination of n/k to obtain the needed spectral range in the selectedfrequency range. An example of these dielectric materials having low nmaterial is SiO², possibly tuned to decrease n even further. An exampleof a high-n material is amorphous silicon, with reduced absorption indexbecause of process parameter tuning, e.g. temperature and hydrogencontent. Hard oxides have better tolerances but cannot be used becauseof the need for higher temperatures than allowed by standard CMOSprocessing.

An example of such alternative mirror system is a (distributed) Braggstack, which is formed by combining two types of dielectrics into analternating stack of two or more materials: one with a low refractiveindex and one with a high refractive index. A first characteristic of aBragg stack is its bandwidth, as given by equation 6, i.e. the spectralrange Δλ₀ over which the reflectivity is more or less constant.

$\begin{matrix}{{\Delta\lambda_{0}} = {\frac{4\lambda_{0}}{\pi}{arc}\sin\left( \frac{n_{2} - n_{1}}{n_{2} + n_{1}} \right)}} & (6)\end{matrix}$

From this equation, it can be seen that the bandwidth Δλo depends onboth the central wavelength λ and the refractive indices n₁, n₂ of theselected materials. To be able to cover a wide spectral range, around acertain central wavelength (e.g. 600 nm spectral range around 700 nm), abig difference between n₁ and n₂ is needed. From the list of materialsthat are used in standard semiconductor processing, SiO² has one of thelowest refractive indices (1:46) and a very low absorption coefficient.Both parameters are stable over a very large spectral range. For aspectral range of 600 nm around a central wavelength of 700 nm (the VNIRrange), this means that the second material in the Bragg stack willideally need to have refractive index equal to 6:4, in addition to anabsorption coefficient as close as possible to 0. There is no such idealmaterial available in the standard IC processing materials, compatiblewith the process flow, and adapting existing materials for a betterrefractive index and lower absorption is needed. The refractive index ofSiO² can be lowered by making it porous (mix it with air, which has arefractive index of 1). This results in a need for better manufacturablerefractive index equal to 5 for the same spectral range and centralwavelength. Another example of material engineering is lowering theabsorption index of amorphous silicon by changing process (deposition)parameters, like temperature, concentration of hydrogen, etc.

$\begin{matrix}{R = \left\lbrack \frac{{n_{0}\left( n_{2} \right)}^{2N} - {n_{s}\left( n_{1} \right)}^{2N}}{{n_{0}\left( n_{2} \right)}^{2N} - {n_{s}\left( n_{1} \right)}^{2N}} \right\rbrack} & (7)\end{matrix}$

As indicated by Equation 7, the reflectivity R of such a Bragg mirror iseasily controlled by the number of pairs of dielectric layers. The morelayers, the higher the reflectivity and the higher the finesse of theFabry-Perot filter that will be built with that particular mirror. InEquation 7, n₀ is the refractive index of the surrounding medium, n₂ isthe refractive index of the substrate, ni is the refractive index of thefirst material, n₂ is the refractive index of the second material and Nis the number of pairs in the Bragg stack. One instantiation of adistributed Bragg stack is a combination of SiO² and engineeredamorphous Silicon for a central wavelength around 700 nm and a rangefrom 540 nm to 1000 nm. A second instantiation is a combination of SiO²and SiGe for a central wavelength of 1500 nm and a bandwidth of 1000 nm,in casu from 1000 nm to 2000 nm. A consequence of using Bragg stacks forthe mirror layers is an additional phase shift during the reflection ofthe light. This phase shift causes the central wavelength to deviatefrom the one given by Equation 3, but this deviation can be easilydetermined using, for example, simulation tools.

Wedge Filter

A wedge filter as shown in FIGS. 6 a-b is an optical filter consistingof a step-like structure. These steps can be ordered to be of increasingheight, in which case they form a monotonic wedge-like structure.However this ordering is not required i.e. non-monotonic structures arealso possible. The filter is post-processed, i.e. monolithic integrated,on top of an image sensor and every step is aligned with a single ofmultiple rows or columns of the image sensor. Every step of the wedgefilters out a different spectral band. As a result, the sensor and wedgefilter combination can be used in hyperspectral imagers of thepushbroom, line scanner type or the hybrid line scanner/starer type.

Existing wedge filters are sloped structures instead of steppedstructures. Due to the lower fabrication complexity, up to now slopedstructures have been used. However, they are only an approximation ofthe desired filter, which should have a constant height for every groupof sensor pixels that are intended to sense the same spectral band. Thisgroup can be arranged as a row or column or any other ordering. In therest of this description it is assumed that the pixels that sense thesame spectral band are arranged as rows or columns. Therefore thepreferred structure of the wedge filter is the step structure, in whicheach row or column of pixels (or groups thereof) is covered with a FabryPérot filter of different height. The staircase structure results indifferent filter properties and a different selected wavelength forevery (group of) row(s)/column(s), in this way resulting in ahyperspectral imager.

The filter of the above can have every step ordered to be of increasingheight in which case they form a wedge. The filter can have the heightof each step being tuned to a filtered spectral band.

The central wavelength of the Fabry-Perot optical filter is determinedusing equation 3 and can be tuned by changing: the length L of thecavity and/or the angle of incidence θ of the light and/or therefractive index n of the material in the cavity

The variable cavity allows building line filters for differentwavelengths by varying the cavity length L over the sensor in onedirection (x or y) while keeping the 2nd dimension fixed. By varying thecavity lengths, it is possible to keep the cavity material (and itsrefractive index) constant. Different lines on the sensor are thensensitive to different wavelengths. Using equation 3, one can calculatethe difference in height H between neighboring lines for a givenspectral resolution. An implementation is given in FIG. 6 , which variesthe length of the cavity linearly over the sensor with the same stepheight between the different steps (note that the height differencebetween the steps is exaggerated for illustrative purposes). The width Wof the different steps then depends on the number of spectralbands/spectral resolution and the width of the sensor.

This embodiment can be easily implemented using binary masks, asexplained later on. One implementation is shown in FIGS. 7 a-7 e , whichillustrate the variation of the selected wavelength over the completesensor. At the left side of FIG. 7 a , the filter characteristic isshown for a line filter with a central wavelength which has very lowtransmission efficiency, because the reflectivity of the used mirrorlayers is not yet optimal in this wavelength range. The centralwavelength gradually increases for increasing wavelengths, with a bigincrease in transmission efficiency at step 20 (around 600 nm), as thereflectivity of the Bragg stacks reaches the targeted performance. Thecentral wavelength further increases until 1000 nm around step 120 witha second order appearing at step 95. The appearance of the second orderis the result of the fact that a Fabry-Perot filter which is designedfor wavelength λ_(j) also passes incoming wavelengths that are amultiple of λ_(j), called higher orders. However, only those higherorder wavelengths that fall in the wavelength range for which both theFabry-Perot filter and the underlying image sensor have a reasonableefficiency should be considered.

The preferred way to change the cavity length will have problems withvariations in the structure introduced during processing. Referringagain to FIG. 7 a , in the VNIR range, the affected wavelength range isapproximately from 800 nm onwards. As a typical CMOS sensor is notsensitive to wavelengths larger than 1000 nm, FIG. 7 a shows a drop inthe transmission above 1000 nm, as only the second order is transmittedand FIG. 7 a shows a selected wavelength that drops back to 700 nm. Thefirst and last region of this wavelength selection seem to capture onlyuninteresting information, as the transmission of the filter in thosewavelength regions is too low or only second order information isrecorded. However, these areas enable the effective spectral range toshift to the left or the right when tolerances are introduced duringprocessing causing a global shift of the filtered wavelength range ineither direction or in variations between different dies.

Manufacturing

Fabrication methods for manufacturing 1D or 2D Fabry-Perot filters caninclude binary or logarithmic construction of the staircase. Astraightforward implementation of the staircase structure, usingsuccessive patterning and etching steps, would require a large number ofprocessing steps in order to produce a staircase with k steps, with ktypically being larger than 50. By using a so-called binary orlogarithmic patterning, the number of required steps can be reduced tolog 2 k, as illustrated in FIG. 5 .

As a result of the binary patterning, e.g. 1024 steps can be constructedby using only 10 patterning steps.

In order to keep the processing costs under control, in particular thenumber of required etch steps to produce the different line filtershaving different thicknesses, techniques like binary or logarithmicmasks can be used as illustrated in FIGS. 9 a-e or in FIGS. 10 a-e . Toillustrate how the different topography is created by inverting masksoptical filters manufactured during the same process steps are given thesame reference number in FIG. 10 e . To reorder the line filters,without increasing the cost of the module, the requirements for thelogarithmic masks should be still fulfilled. However, it is possible toachieve this reordering, or approximate it very closely while stillachieving the target, by using the inverse of some of the normal binarymasks, as will be obvious to those skilled in the art and only use 1Dreordering However, this simple reordering of the steps will only affectthe angle of incidence in the direction perpendicular to the linefilters. In practice, this will lead to a more balanced and highersensitivity for the sides of the wavelength range, but only in themiddle of the sensor. This can still be useful for many applications,especially when the region of most interest is the center of the scannedimage.

In order to compensate for the second dimension, i.e. along the lengthof the line filter, additional process steps are required, which willintroduce a trade-off with cost. If the additional cost can bemotivated, the average angle dependence can be compensated for by addingan additional variation in the second dimension. By varying the cavitylength in the direction of the line filter, the effect of the variationin average incident angle can be minimized. For this technique,additional etching steps are required and the final cavity will have avarying thickness parallel to the length of the line filter.

In the present application non-monotonically rising (or sinking) wedgesare used in order to absorb errors in the etching. The non-monotonenature can come from the fact that we etch one step of the wedge lesslong than necessary which results in the step being higher. Thestructure is thus continuously falling except for a few places. See alsoexamples in FIG. 24, 25 or 26 described below.

What is also useful is that we can also configure the patterning of themasks to achieve beneficial effects. See for example translating and/orinverting, as the result of which we get other types of wedges.Example—see the “hill” profile as shown in FIG. 25 . This is a structurethat is useful to compensate on the sensor for light fall-off. At theedges of the senor, you sometimes get vignetting which means a lowerintensity of the light. On these places lie the regions of our wedgewhich have minimal sensitivity on the sensor, namely 400 nm and 1000 nm.Using the “hill” we can compensate for this. We match the sensitivity ofthe sensor to the sensitivity of the optical components which lie beforeit.

Particular examples of a complete HSI camera comprising themonolithically integrated subsystem are given later in this description.Designing a Fabry Perot for first order forces strict tolerances on thethickness of the several layers (in nm range), which are difficult toachieve with our low-cost processing flow (hardness, oxide, and so on .. . ). State of the art wedges (LVF and staircase) have increasing oreven monotonically increasing thickness in one direction. Additionalsteps are needed to make this design more tolerant for processvariation:

Manufacturing for Processing Variations and Tolerances

As the processing technology requirements on the dimensions of the wedgefilter are very strict, variations on step height, width, placement,corner sharpness and orientation can be expected. The design will besuch that the nominal design targets a wider range of wavelengths thanis required by the targeted applications, which corresponds to theinsertion of extra steps into the wedge filter as shown in FIG. 12 . Asa result, a deviation of the produced height with respect to the nominaldesign, will cause a shift in the effective filter range to either sideof the sensor. Through the insertion of the extra steps on each side,the wavelength range that is requested can still be recovered by readingout different columns of pixels (after a post-production calibrationstep).

The hyperspectral imaging filter set has been designed taking intoaccount that various processing steps will always have tolerances asshown in FIG. 11 . These tolerances occur on all normal steps of thefabrication and controlling them is usually a cost trade-off. Everyprocessing step can be controlled up to a certain extent and this can beimproved by investing in extra process development and refinement, up toa certain extent. As a result, for many steps it is very difficult toquantify the exact limits on these variations. The philosophy behind thecurrent design is to prevent the expensive and time consumingoptimization steps as much as possible by taking into account thesevariations, if the effect of the variation can be overcome by softwarecorrections or modifications to the design. By taking this approachduring the design, it is possible to propagate requirements down to theprocessing steps: taking the slack that is created by the tolerantdesign into account, the variations that occur in the processing stepsshould stay below a predefined threshold. This threshold is set atdesign time, based on the expected variability in the various processingsteps. Variations above the threshold can no longer be compensated forand will result in modules that do not meet the specifications. Thefollowing sections introduce briefly some of the process steps and inwhich way they introduce a certain amount of variation and their effecton the filter structure as a result of the tolerances of that step.

Different types of tolerances or variations exist. The across the wafervariation (inter-die) will have different implications on the finaldevice than the intra-die variations. In the following text, both arecovered in general as tolerances or variation, unless specifiedotherwise.

The rationale behind the systems design is that the different filterlines of the hyperspectral imaging module will sample different pointsin the spectrum at a certain sampling interval. A first choice is tosample the spectrum with maximal spectral resolution. This rate can bederived from the rayleigh criterion, which states that two filters arespectrally resolvable if they cross in their 3 dB point. A second choiceis to sample the spectrum at a reduced rate, e.g. Shannon's rate tocover all frequencies in the signal. In the latter situation, smallvariation in the effective filter location will not have an importanteffect on the use of the module to sample the spectral curve. Onlyapplications that aim to detect very narrow spectral peaks at a certainwell defined wavelength will suffer from the variations. This sectiondiscusses the several causes of these variations and the techniques thatwe apply to cope with them.

Planarity of the Image Sensor

In order to start with a well controlled state, it is important that theimage sensor is planarized before the filter structure is built up. Thiscan be done using a deposition step, followed by a CMP (ChemicalMechanical Polishing) step to remove all topography. By doing this, therest of the processing does not depend anymore on the exact BEOLarrangements. The thickness and the material of this planarization layercan to some extent be taken into account during the design of the filterstructure. However, this layer is not a part of the active filterstructure and does not have a large effect on the filter itself, as longas the correct material transition (important for the refractive index)is correctly taken into account. As the Fabry-Perot filter will bedeposited on top of this planarization layer, variation in this layerwill be not propagated up, as long as the variation is sufficiently slowacross the wafer (e.g. no sharp edges). As CMP is able to generate asurface with across wafer flatness and variations at the nanometerscale, this requirement can be fulfilled.

Deposition Tolerances

A variation in deposited thicknesses in the components of theFabry-Perot filters, in case the layers of the Bragg stack and thethickness of the cavity, will result in a mismatch between the designedfilter and the produced filter. The effect of the variations on thethickness of the cavity is that: the thickness of all filters will bechanged by more or less an equal amount, causing a shift of the spectralrange to the right of the left of the theoretical design. This globalshift in the selected wavelengths, either up or down, with respect tothe designed filter location, can be compensated for by extending therange. By adding additional filter structures that cover a safety zoneon either side of the desire spectral range, the tolerance on thedeposition of the cavity can be covered. E.g. if the total variabilityon the deposited height of the cavity is maximally 20 nm, this can berelated to the number of additional steps that has to be added. For aspectral different of e.g. 5 nm between, the cavity could be 10 nmhigher or lower than in the design, leading to a modified range designincluding 2 additional steps, both for the smallest cavity length andfor the biggest cavity length. This can be linked to the actual designby combining the needed number of spectral bands, the number of spatiallines under one band and the size of the sensor. This determines thefree area on the sensor used as input to calculate the amount of extrabands and thus the maximum allowed variations on the deposition.

FIG. 12 shows a schematic representation of a hyperspectral imagingmodule for which the line filters are ordered from λ_(j) to λ_(j+k),e.g. from blue to red, and for which on both sides extra line filtershave been added. These filters will not be used for the nominal point,when the design is produced without significant variation in thedeposition of the cavity. However, when the initial cavity deposition isoff, either side of the extra line filters will fall inside the intendedrange and will be functional, while more filters on the other side willbe disabled. This range shift can easily be calibrated post-fabrication,by illuminating the full filter structure with some known wavelengthsand by storing the location of the line filter with the highest responsein a memory.

In addition to the wafer-wide deposition tolerance, designing anextended range, coupled to calibration, will also cover the expectedinter-die variation. When needed, some additional steps can be added tocover this type of variation, or traditional binning, selecting certaindevices for certain wavelength ranges. Intra-die variation cannot behandled by adding more steps and therefore the intra-die variationshould be limited and should be less than the difference between twosteps (e.g. 3 nm). If the intra-die variation exceeds this difference,the difference between two line filters with minimal nominal wavelengthdifference is no longer defined. Intra-die variations are smaller thanvariations inter-die variations (across the wafer). Variations acrossthe wafer cause shifting of spectrum in one direction. Extended range isforeseen to cope with this shifting.

Etch Tolerances

After the initial deposition of the cavity material for the Fabry-Perotfilter, different filter instances, e.g. for different line filters, canbe made by etching this cavity material. The resulting Fabry-Perotfilters will be defined by their respective different cavity heights.The exact wavelengths response of each individual line filter willdepend on the target height and the process tolerances of the variousetch steps by which the final height of the step is defined. To reducethe total number of etch steps that are required, techniques like binarymasks or logarithmic masks can be used, by which only n etch steps arerequired to fabricate 2 n different cavity heights. As was discussedabove, the cumulative variation on the different etch steps that arerequired to define a certain target cavity length should be limited andless than the difference between two steps (e.g. 3 nm). However, somedesign tricks can be applied to stretch this requirement. If the opticalfilter is now designed that this tolerance is completely covered byintroducing overlap, i.e. several parts of the mask contain the samewavelengths as shown in FIG. 13 , the correct wavelengths can beallocated using a calibration and software processing.

In case the etch processes that are being used to define arenon-directional processes, the sharp edges that form the transitionbetween one line filter and the next one, can show rounding. In thepresented embodiment, the width of each line filter can cover multiplecolumns of pixels. In case the post-production characterization showssignificant distortions of the filter, due to corner rounding, theaffected columns can be disabled or removed in software post-processing.This is a form of redundancy and it is part of a trade-off between thecost of process optimization and the performance of the produced device,in this case a reduction in the number of used columns of pixels. Asindicated above section, a filter can be designed for a minimum numberof spectral bands so that Shannon's sampling law is not broken. This canthen be used to e.g. reduce the number of layers in the distributedBragg stacks to reduce the Finesse and thus increase the FWHM of theFabry-Perot filter. However, small variations on etching will cause thefilters to shift a little bit to the right or the left. In both casesinformation from the spectrum will be missed. Spectral oversampling is atechnique that uses the Fabry-Perot optical filter at its maximum FWHMto make it maximally spectral resolvable. The FWHM of these filters canbe increased using system techniques. The additional spectral bandsintroduced by the spectral oversampling will overlap partially with theoriginal ones, but they will make sure that all relevant information isacquired.

Non-monotonically increasing filters provide for spectral overlap insame wedge selected to give redundant information for most critical etchsteps. Spectral oversampling takes care of shifting wavelengths in onedie: FWHM designed to be smaller than needed for spectral resolution.The number of bands is calculated using the given FWHM to cover completespectral range. System aspects, e.g. having smaller fo, will cause FWHMto increase, hence neighboring filters will start to overlap and formone filter for 1 spectral resolution. Due to oversampling, all spectralinformation will be sensed and can be extracted using calibration andstandard image processing.

Alignment Tolerances

When using standard IC processing techniques, alignment of filterstructures on top of rows/columns of pixels with dimension of severalmicrons per pixels is well within the possibilities of the state of theart. Therefore, alignment at the top level is not very critical. Asdiscussed in the previous paragraph, when a misalignment would occur, asa single line filter can cover multiple columns of pixels, the offendingcolumn can be disabled. Again, this is part of the same trade-off.

Design for Optical Fall-Off and Module Sensitivity

When designing the hyperspectral module, consisting of both the imagesensor and the filter structure, cross-component optimizations can bedone. As the proposed hyperspectral module is targeting low-cost and/orcompact systems, lower quality optics can be expected. One effect whichcan be tackled in this context, is vignetting. Vignetting is a reductionof an image's brightness or saturation at the periphery compared to theimage center. When this effect is coupled to the wavelength dependentefficiency of the Fabry-Perot filter and image sensor, both effects canbe co-optimized in order to flatten the wavelength dependent behavior,instead of strengthening it.

As vignetting causes the light intensity to drop from the center towardsthe sides of the image, the effect for a scanning application can besplit into two components. The effect of the intensity fall-offperpendicular to the scanning direction can be compensated for by theillumination, as is known to those skilled in the art, by the use ofso-called illumination profiles. In the scan direction, a secondopportunity exists, by exploiting the potential to reorder the linefilters in such a way that the sensitivity/intensity difference is levelout, instead of strengthened. Image sensors are designed for a certainwavelength range. E.g. CMOS imagers can in most cases be used in the 400nm to 1000 nm range. However, the efficiency of the sensor is not thesame over the complete range.

Both effects, vignetting and sensor sensitivity, affect the efficiencyof the module for a certain arrangement of the line filters. When astraight forward ordering of the line filters, monotonously increasingin target wavelength, e.g. from 400 nm to 1000 nm in 10 nm increments,is chosen, the areas of the sensor (in the scan direction) that areaffected most by the vignetting are the top and bottom most filterlines. For the straight forward ordering, these are the filter lines forthe wavelengths that the sensor is least sensitive to. Hence, botheffects add up and the hyperspectral module will have a suboptimalsignal to noise ratio at the sides of the targeted wavelength range. Inorder to flatten the sensitivity and overcome this additive behavior ofboth effects, a reordering can be done that takes both effects intoaccount. FIG. 15 show a schematic representation of a hyperspectralimaging module in which the line filters are no longer monotonouslyincreasing, but for which the filters have been reordered. Filters thatselect wavelengths for which the sensor is the least sensitive areplaced in the middle of the sensor, where no (or the least) vignettingwill occur. Hence, both effects work in the opposite direction and theefficiency across the complete spectral range is flattened. This can becombined with illumination profiles, if needed, and when the applicationpermits.

As has already been discussed in the foregoing paragraphs, one part ofthe design of the hyperspectral imaging module, is the distribution orordering of the different line filters over the image sensor. Ingeneral, the design can be split into the following parts:

1. selection of the targeted wavelength ranges

2. selection of an image sensor for that range

3. selection of the targeted spectral sampling (and spectral resolution)

4. design of the different Fabry-Perot line filters

5. ordering of these Fabry-Perot filters over the image sensor

The ordering of the filters, in principle, does not matter, as thedifferent filtered wavelengths can be regrouped into a hyperspectralimage in software after the scanning, whatever the ordering would be. Amethod to tolerate process technology variations is applied by makinguse of a filter as defined in above, the wavelength range beingrecovered after a post-production calibration step by reading outdifferent rows or columns of pixels covered by a filter of the sameheight. However, still different types of ordering make sense, whentaking into account other systems aspects, like production cost,sensitivity etc.

The first and most straight forward ordering, is called the wedgeordering, as its shape at the abstract level resembles a wedge or moreaccurately a staircase. In this ordering, all line filters are orderedaccording to a monotonously increasing filter wavelength. A graphicalrepresentation of the wedge ordering is shown in FIG. 12 . Oneextension, as already discussed before covers a repetition of certainline filters in the staircase structure, in order to cover thetolerances in the processing. If certain critical etch steps wouldoveretch, some sampling points in the hyperspectral image would bemissing. In order to prevent this, a deliberate design modification ismade, that intentionally creates a non-monotonously increasing staircasestructure. At some critical points, the design will foresee a repetitionof some line filter that can then later be removed in post processingbut even when the processing tolerances would tend to over etch, nosample points would be missing. A graphical representation of thisconcept is shown in FIGS. 13 and 14 , that clearly shows the overlappingrange is the middle of the image sensor. As the ordering no longermonotonously increasing, technically this no longer considered to be awedge.

System Aspects to Maximize Optical Throughput, Resulting in an IncreasedSpeed:

Avoid the use of additional filters for order removal, use imageprocessing for order removal;

Eliminate slit and collimator increases the optical throughput in thissystem, but FWGM increases and spectral resolution decreases: spectraloversampling allows this; and

Spectral oversampling enables a more open aperture (replacing slit) forincreasing optical throughput.

As discussed above, a Fabry-Perot Filter is sensitive to the angle ofincidence of the light onto the filter: both the central wavelength asthe FWHM depend on this incident angle. A special optical configurationcan be used to minimize the impact of this dependency on the overallperformance, e.g. when multiple filters are combined into a filtermodule. This section discusses this optical system and the trade-offsthat impact the optical throughput, spectral resolution, full width halfmax (FWHM) of the filter, etc. A first system, discussed below, achievesthe best spectral resolution and FWHM, but at the lowest opticalthroughput. Opening up the stop, by replacing the slit with a variableaperture, improves the optical throughput, but worsens the FWHM andspectral resolution. This trade-off is discussed.

The integrated wedge filter can be used in different systems setups.Depending on the system integration, the resulting performance of thewedge filter, both for speed as for spectral resolution is different.One important aspect of the optical system, for which one example isshown in FIG. 18 is the size of the exit pupil.

The size of the exit pupil has a direct effect on the size of thevariation in the angle of incidence of the light onto the Fabry-Perotfilter that is formed by each step of the wedge. For a pixel p, below agiven step of the integrated wedge filter at a distance x from theoptical axis, the angles between the incident ray parallel to theoptical axis and the top and bottom of the exit pupil are called α and βrespectively.

As can be seen from FIG. 4 the size of α and β depend on the size of theexit pupil. Since the selected wavelength of the Fabry-Pérot filterdepends on this incident angle, this results in various wavelengthsbeing selected by each step of the integrated wedge filter. Thisrelation is described by the following three equations.

$\begin{matrix}{{\alpha = {{{{atan}\left( \frac{{D_{i}/2} - x}{i} \right)}{and}\beta} = {{atan}\left( \frac{{D_{i}/2} + x}{i} \right)}}}{\theta = {{{{asin}\left( \frac{\sin(\alpha)}{n_{cavity}} \right)}{and}\theta_{2}} = {{asin}\left( \frac{\sin(\beta)}{n_{cavity}} \right)}}}} & {{Eq}(8)}\end{matrix}$λ_(α)=λ·cis(θ₁) and λ_(β)=λ·cos(θ₂)

At the systems level, the direction of the incident light can becontrolled by the use of a collimator and/or a telecentric lens. Thefollowing paragraphs describe different system integrations, both withand without collimator. Depending on the application, both have adifferent improvement over the current state of the art solutions.

Collimated (can be Used for Higher Spectral Resolution than GratingBased Systems)

This subsection describes the possible optical system setups to use theproposed filter module as a pure line scanning hyperspectral camera. Inthis setup, all wavelengths for a single line are collected at the sametime. As illustrated in FIG. 1 the hyperspectral image cube is thenconstructed scanning the scene line per line. An objective lens for theimage forming is used and a slit for selecting a single line from thisimage. The collimator is used to control (minimize) the angle ofincidence of the light rays onto the optical filter or the imager. Atthe output of the collimator the light rays are nearly parallel. Becauseof the well selected location of the slit on the optical axis and thevery small size of the slit, these light rays are parallel to theoptical axis. The collimator is a plano-convex lens and is not arotational symmetric lens. Its collimating function is restricted to thedirection of the shown cross-section. In the perpendicular direction,the direction of the slit, there is no collimating effect. As a result,the image line selected by the slit is duplicated over the completesensor with an angle of incidence perpendicular to the optical filter.Consequently, the light energy in that line is also spread over thecomplete sensor. Light rays that originate on image lines above or underthe optical axis (e.g. by widening the slit) will also be parallel aftercollimation, but will not be parallel with the optical axis. Thespectral resolution for a system using a collimator will therefore beindependent of the f/# of the objective lens, but will be dependent onthe slit size.

The first set-up, as shown in FIG. 19 and FIG. 30 is a line-scannerconsisting of an objective lens 82, slit 83, collimator 85 andintegrated circuit 5 having the wedge filter on top of a standard imagesensor. As a result of the use of the collimator, the angle of incidenceof the light on top of the wedge filter is controlled well and thisresults in a good spectral resolution. However, due to the use of aslit, the amount of light (and hence the amount of energy) that entersthe system is heavily reduced. This results in larger integration timesfor the sensor and in an overall reduced speed. FIG. 19 details thecollimated system. The scene, at distance 0, is imaged by the objectiveonto the slit, at distance dslit. The focal points of the objective areindicated by two points f. The Numerical Aperture NA is related to theamount of light that is passed by the objective lens. The light thatpasses through the slit falls onto the collimator, at distance fcol. Thecollimated light is then projected onto the filter and sensor, atdistance dwedge. The effect of the slit and collimation on the spectralresolution is shown in FIG. 20 , where the angle θ is proportional tothe height of the slit Yslit and represents the deviation of theselected wavelength for α and β with respect to the nominal wavelength.

From FIG. 21 it can be seen that the deviation of λ_(a) and λ_(β) withrespect to the nominal wavelength result in a relative deviation that isrelated to the slit size and the object distance. For a slit size of 80μm, the relative deviation is still below one pro mille for a collimatorwith a focal length of over 10 mm. For smaller slit sizes and largerobject distances, this deviation is even smaller. As a result, very goodspectral resolutions can be achieved with this system.

The achievable spectral resolution is in this system better than for agrating based system, as the limiting factor in grating based systemsdepends on the dispersion per pixel and the higher sensitivity of thegrating equation (equation 10) to changes of the incident angle comparedto the Fabry-Perot equation (equation 9). For the angle of incidence ofinterest (θ=0), the sensitivity of the grating equation is maximal,while for the Fabry-Perot equation this sensitivity is minimal.

$\begin{matrix}\left\{ \begin{matrix}{{m\lambda} = {2{nl}\cos\theta}} \\{\frac{d\lambda}{d\theta} = {2{nl}\sin\theta}} \\{\theta = {{0\frac{d\lambda}{d\theta}} = 0}}\end{matrix} \right. & (9)\end{matrix}$ $\begin{matrix}\left\{ \begin{matrix}{{m\lambda} = {2{nl}\cos\theta}} \\{\frac{d\lambda}{d\theta} = {2{nl}\sin\theta}} \\{\theta = {{0\frac{d\lambda}{d\theta}} = 0}}\end{matrix} \right. & (10)\end{matrix}$

In addition, the width of the spectral band for a grating based systemalso depends on its dispersion per mm, which depends on the pitch p.Because of this continuous dispersion, a complete spectral band isprojected on a single pixel. The larger the area of the pixel, thefaster the sensor, but the higher the width of the spectral band willbe. The width of the spectral bands of the proposed wedge filter isindependent of these pixel sizes and only depends on the materialparameters.

Due to the use of a slit, however, the amount of light that enters thesystem is heavily reduced. This is expressed through the opticalthroughput, which is a geometric measure of how much light is allowed toenter the optical system. Because slits have a significantly reducedarea, the optical throughput of these systems is also drasticallyreduced, limiting the amount of light that can enter in the camera andthus limiting the speed of the camera.

Un-Collimated (can be Used for Faster Systems than Grating BasedSystems)

An alternative system setup does not have a slit and projects a completeimage frame onto the sensor. As shown in FIG. 31 , there is an objectivelens 82 and the integrated circuit 5. The sensed image will thereforerepresent all the spatial information in the object, but as thedifferent lines on the sensor are sensitive to different spectral bands,the different lines in the image will also contain information fromdifferent spectral bands. Collecting all the spectral bands for a lineis done by scanning the line over the sensor and subsequently combiningall spectral information corresponding to the same spatial line fromdifferent frames into one hyperspectral image cube. E.g. assume anobject for which a first line is projected on the first line of thesensor, sensitive to one specific spectral band b1. The first image linewill therefore only contain information of this band b1. Next, the lineof the object moves to the second line on the sensor, which is sensitiveto another spectral band b2. The second band for that line will then becollected at the same time as the first band is collected for the nextline on the object. This procedure is then repeated until the completeobject is scanned in all wavelength bands.

The second system integration option uses no slit or collimator and theintegrated wedge filter with sensor is combined with an objective lensinto a system. By eliminating the slit and collimator, the total systemcost is reduced and the amount of light that enters the system isincreased, which can lead to a faster camera. However, the angle ofincidence of the light onto the different filter steps of the integratedwedge filter is less controlled, which results in a reduced spectralresolution if the lens system is not carefully designed.

As was shown in FIG. 4 , the angle of incidence depends heavily on theexit pupil of the objective and hence on the aperture. FIG. 22illustrates the effect of the objective aperture on the worst case rangeof wavelengths that are being selected by the steps of the wedge filter(with Lambda 1=α and Lambda 2=β). In FIG. 22 a a large aperture (f1.65),which corresponds to a fast system, results in a spectral resolutionthat will be no better than 60 nm from 400 to 800 nm and even lessbetween 800 and 1000 nm. However, by reducing the aperture (f22), asshown in FIG. 22 b the spectral resolution can be increased and aresolution of about 15 nm can be reached across most of the range ofinterest. However, this again results in a loss of light and hence aslower system.

Careful lens design is thus needed to maximize the numerical aperture(optical throughput) and optimize the spectral resolution. One example(but not limited to) of such a lens is a telecentric lens, which is alens with the chief rays in parallel with the optical axis. These lensessignificantly limit the angle of incidence of the light and are perfectcandidates as objective lenses for these cameras.

FIG. 23 , Integrated Circuit According to an Embodiment Having ParallelRead Out

FIG. 23 shows an integrated circuit having optical sensors 40 in groups20 underneath optical filters 10 of differing thicknesses. Read outcircuitry 30 has output circuits A and B for each of the groups, (thoughoutput circuits are shown only for one group for the sake of clarity) sothat an image having various spectral bands can be output (multi lambdaimage). For each group, some of the optical sensors are coupled tooutput circuit A and others to output circuit B. Of course, there may bemany more. In some cases, these may be one output circuit per sensor, toprovide more complete parallelism in the read out. This enables theoptical sensors for one group to be read out in parallel and thus readout more quickly, or a larger group to be read out in a given time. Thegroup can be a line, or any other shape. The parallel outputs can beoutput in parallel, or can be multiplexed before leaving the integratedcircuit. The optical sensors for each output circuit can be interleavedwith those of other output circuits, or be in concatenated sections of aline for example.

FIGS. 24-26 , Embodiments Having Non Monotonic Thicknesses

FIG. 24 shows an integrated circuit having optical sensors 40 in groups20 underneath optical filters 10 of differing thicknesses. Read outcircuitry 30 is provided so that an image having various spectral bandscan be output (multi lambda image). In this case the thicknesses vary soas to increase and decrease across the array, rather than varyingmonotonically.

FIGS. 25 and 26 show examples of other profiles of thickness across thearray of optical filters. FIG. 9 shows a peak near the middle of thearray. FIG. 10 shows a saw tooth arrangement (steps too small to beresolved in this view) with clusters of optical filters havingoverlapping spectral bands. This gives some redundancy which can beexploited in later image processing to enable more tolerance ofimprecision in the manufacture of the optical filters.

FIGS. 27-29 , Embodiments Having Lambda Selection.

FIG. 27 shows an integrated circuit according to another embodimenthaving optical sensors 40 in groups 20 underneath optical filters 10 ofdiffering thicknesses. Read out circuitry 30 has output circuits C and Dfor different optical filters (in a cluster of two or more of suchfilters), and a lambda selector 50 arranged to select either of theseoutput circuits or interpolate between them, so that an image havingvarious spectral bands can be output (multi lambda image). This canenable spectral sampling or spectral interpolation, which again canenable greater tolerance of errors in thickness of the optical filters.If the clusters effectively overlap with each other so that some opticalfilters belong to two clusters then the wavelength selectors can becontrolled to effectively shift the wavelength without necessarilysubsampling.

FIG. 28 is similar to FIG. 27 , but with multiple output circuits foreach group, which can be arranged to read out in parallel and feedparallel or multiplexed signals to the lambda selector to improve readout speeds for example.

FIG. 29 is similar to FIG. 27 or 28 , but with the lambda selector nowimplemented off chip, as a function of an image processor 53. This canenable the integrated circuit to be simpler, but may involve higher datatransmission rates off the chip.

The integrated circuit can be approximately 1 cm square for example. Itcan have a standard array of optical sensors (FSI) on one surface ofwhich is formed a bottom semi transparent mirror of Al, after aplanarization and/or anti reflective coating has been applied. Thetransparent layer in the wedge shape can be formed of SiO². As discussedabove, the wedge need not have a monotonic change in thickness acrossthe array. A top semi transparent mirror can be formed of a layer of Al.Each of the manufacturing steps can be implemented using various knowntechniques.

Summary of Some Additional Features:

The integrated circuit can have each of the optical filters having alayout as a strip across the integrated circuit, the group of sensorsfor a respective one of the optical filters having a layout extending asa corresponding strip. The group of sensors can have a layout configuredas two or more lines of sensors corresponding to the layout of thestrip, each of the lines of sensors being coupled to a different one ofthe output circuits.

The read out circuitry can have a wavelength selector for selectingbetween or combining (such as by interpolating between, or othercombination of) read out signals of corresponding pixels of differentoptical filters to tune the output to correspond to a particularwavelength. The locations of the optical filters can be arranged so asto have optical filters for wavelengths for which the optical sensorsare less sensitive located at locations where the incident illuminationwill have lower intensity. Typically this is near the centre and awayfrom edges of the sensor array. The locations can be arranged to providea cluster of adjacent optical filters having different thicknesses toenable detection over a first spectral band, and to provide aneighbouring cluster having different thicknesses to enable detectionover a second spectral band such that the first and second spectralbands overlap.

At least some of the sensors can be arranged in groups each receivinglight from a corresponding one of the optical filters, and the read outcircuitry can comprise at least one output circuit coupled to thesensors of a respective one of the groups, with a wavelength selectorfor selecting between or interpolating between read out signals ofdifferent groups corresponding to different ones of the optical filters,to provide an output for those groups tuned to correspond to aparticular optical wavelength.

The locations can be arranged to provide a given one of the opticalfilters with neighbouring strips which are both thicker or both thinner,to provide a valley or ridge structure respectively. (Valleys/ridges canbe local or across part or all of array. Non-monotonic can also be arandomized ordering, such that valleys/ridges are too short to appearany more.) The thicknesses of the optical filters can be configured tocompensate for differing angle of incidence of light at differentpositions across the array of optical filters. In the example of alongitudinal strip filter, the angle of incidence is likely to begreater near the extremities and so the thickness should be less, sothat the path length is constant.

The locations can be arranged to provide a cluster of adjacent opticalfilters having different thicknesses and the read out circuitry having awavelength selector for selecting between or interpolating between readout signals of corresponding pixels of different optical filters of thecluster to tune the output to correspond to a particular wavelength.

The wavelength selector can be arranged to output signals representingproportionately fewer wavelengths than the quantity of different opticalfilters provided on the array of optical filters, so as to provide aproportionate spectral subsampling.

The wavelength selector can be arranged to output signals representing anumber of wavelengths similar to a quantity of different optical filtersprovided on the array of optical filters, so as to provide a spectralshift.

The amount of spectral shift can be varied according to location in thearray of optical filters to compensate for manufacturing variations inoptical filter thickness at different locations.

An imaging system can have the integrated circuit and an external imageprocessing part coupled to receive the pixel values representing theimage, and to output an image processed version of the received image.The imaging system can be arranged to generate and store an image cubeof an object, by relative motion of the integrated circuit and theobject being imaged, the image cube having x and y spatial dimensions,and a spectral dimension. The imaging system can be arranged to apply alambda selection or interpolation image processing function, tosubsample the image cube in the spectral dimension or to shift the imagecube in the spectral dimension. The variation according to location canbe suitable to compensate for any one or more of the following:manufacturing variations in optical filter thickness at differentlocations, distortions owing to variation in angle of incidence of anoptical path through the optical filter, higher order removal anddistortions from other optical components.

The imaging system can have any one or more of: an objective lens a slitand a collimator, in an optical path leading to the array of opticalfilters. At least some of the optical filters can have a thicknesssuitable to distinguish higher order interference, and the imageprocessor can be arranged to compensate for higher order interferenceeffects in the rest of the image representation according to an amountof higher order interference distinguished by those optical filters.

Some embodiments have an optical filter array with a stepped-likestructure post-processed on top of an image sensor array, the filterbeing positioned in direct contact with the image sensor array. Thefilter array in some cases has every step of the filter aligned with asingle row or column of pixels of the image sensor array. In some caseseach row or column of pixels is covered with a Fabry Pérot filter ofdifferent height. In some cases the changes in thickness are monotonic,and in other cases they are non monotonic, to create ridges or valleys,across the array. In other cases, a thickness of each step is tuned to afiltered spectral band.

Methods for manufacturing monotonic or non monotonic changes in thethickness of the filter array can include using a binary or logarithmicpatterning technique.

Examples of a complete HSI system can include an image processor afterthe optical sensor array, and optical parts before it.

A hyper spectral camera system can consist of an optical filter arraypost-processed on an image sensor array as defined in the above, thesystem further comprising an objective lens and/or slit and/or acollimator.

An effect of the non monotonic variation of thicknesses is to reorganisethe relationship between frequency and the differing sensitivities ofparts of the image sensor arrays (place high or low frequency in themiddle instead of on the edge of the sensor). Also it can reduce thesensitivity of the filters to processing variations and hence canincrease yield. It can enable several differing wavelengths clusteredabout a one wavelength to be received by different ones of the sensorsand then the wavelength to be selected or processed later that is bestsuited. Non monotonic variation enables grouping of some spectral bandsin a cluster (range) and to position them arbitrarily on the sensorarray. This allows many things, like reordering for tolerances,compensating for fall-off, and so on. Monotonic wedges can do some ofthis but will however never be able to cope with tolerances because ofetching, whether they are larger or smaller than the depositiontolerances doesn't matter. The non monotonic variation enables the intracluster variation to be greater than the inter cluster variation. Inanother example an increase in width of 1 or more steps can be providedfor the most important bands for the particular application. Anotheralternative is to adapt the ordering of the different steps to matchareas of maximum sensitivity of the optical parts to less sensitiveareas of the sensor. So the middle of the sensor array can be used forthe most important bands for the application, by making filter arrayhave the appropriate thickness in the middle for example.

An effect of selection according to calibration input is that thecalibration input can compensate for process variations either intra-dieor inter-die variations across a wafer of many dies, or even inter wafervariations, if there are enough sensors and optical filters toeffectively oversample the spectrum, or to have extended range so thatselector selects the best suited filters that most closely match adesired set of wavelength values.

Typical System Tradeoffs:

For line scanning imagers, a good spectral resolution is typicallyobtained through the combined use of the slit and the collimating lensEliminating these parts can cause a decreased spectral resolution.Indeed, the slit and collimating lens control the angle of incidence ofthe light on the sensor, which in many wavelength selectors is animportant parameter. The spectral resolution is known to vary as afunction of the angle of incidence on the sensor. However, theelimination of the slit increases the optical throughput and thusincreases the speed of the system;

The integration of the wavelength selection component on top of theimager not only reduces the amount of stray light (increasing thespeed), but also enables a reduction in the cost of the system; and

The co-design of the wavelength selection component with low-level imageprocessing can enable larger tolerances on the wavelength selector.

Furthermore, by providing application dependent image processing, adrawback with current hyperspectral imagers that are typically researchinstruments, with image processing delivered as a research instrument inpackages that are typically only useable by experienced and trainedpeople on high performance infrastructure, can be overcome. Real-timehyperspectral image processing can enable use of such hyperspectraltechnology in industrial machine vision and medical imaging amongstothers.

Processing Hardware:

Some of the method steps discussed above for image processing forexample, may be implemented by logic in the form of hardware or, forexample, in software using a processing engine such as a microprocessoror a programmable logic device (PLD's) such as a PLA (programmable logicarray), PAL (programmable array logic), FPGA (field programmable gatearray).

An example of a circuit with an embedded processor may be constructed asa VLSI chip around an embedded microprocessor which may be synthesizedonto a single chip with the other components. Alternatively othersuitable processors may be used and these need not be embedded, e.g. aPentium processor as supplied by Intel Corp. USA. A zero wait state SRAMmemory may be provided on-chip as well as a cache memory for example.Typically I/O (input/output) interfaces are provided for accessingexternal storage e.g. via data networks. FIFO buffers may be used todecouple the processor from data transfer through these interfaces. Theinterface can provide network connections, i.e. suitable ports andnetwork addresses, e.g. the interfaces may be in the form of networkcards.

Software:

Software programs may be stored in an internal ROM (read only memory)and/or on any other non-volatile memory, e.g. they may be stored in anexternal memory. Access to an external memory may be provided byconventional hardware which can include an external bus interface ifneeded, with address, data and control busses. Features of the methodand apparatus of various embodiments may be implemented as software torun on a processor. In particular image processing in accordance withcertain embodiments may be implemented by suitable programming of theprocessor. The methods and procedures described above may be written ascomputer programs in a suitable computer language such as C and thencompiled for the specific processor in the embedded design. For example,the software may be written in C and then compiled using a knowncompiler and known assembler. The software has code, which when executedon a processing engine provides the methods and image processor forcertain embodiments. The software programs may be stored on any suitablemachine readable medium such as magnetic disks, diskettes, solid statememory, tape memory, optical disks such as CD-ROM or DVD-ROM, etc. Othervariations can be envisaged within the claims.

The foregoing description details certain embodiments of the invention.It will be appreciated, however, that no matter how detailed theforegoing appears in text, the invention may be practiced in many ways.It should be noted that the use of particular terminology whendescribing certain features or aspects of the invention should not betaken to imply that the terminology is being re-defined herein to berestricted to including any specific characteristics of the features oraspects of the invention with which that terminology is associated.

While the above detailed description has shown, described, and pointedout novel features of the invention as applied to various embodiments,it will be understood that various omissions, substitutions, and changesin the form and details of the device or process illustrated may be madeby those skilled in the technology without departing from the spirit ofthe invention.

What is claimed is:
 1. A method for calibrating an image sensor,comprising: illuminating at least a portion of the image sensor with aninput light spectrum, wherein the input light spectrum includes light ofknown wavelength and intensity, wherein the image sensor includes aplurality of optical sensors; sampling an output for each optical sensorof a plurality of optical sensors of the image sensor, wherein eachoptical sensor is associated with one or more optical filters, whereineach optical filter is associated with a group of optical filters of aplurality of groups of optical filters, wherein each optical filter of agroup of optical filters is configured to pass light in a differentwavelength range and wherein at least some optical filters in differentgroups of the plurality of groups of optical filters are configured topass light in substantially a same wavelength range; comparing a sampledoutput for each optical sensor of the plurality of optical sensors withan expected output; based on the comparing, generating a calibrationfactor for each of at least a subset of the plurality of opticalsensors; and storing the calibration factor for each of the at least asubset of the plurality of optical sensors in memory.
 2. The method ofclaim 1, wherein the optical filters of each group of optical filtersare further configured to provide a spectrum of wavelengths.
 3. Themethod of claim 1, wherein each group of optical filters of theplurality of optical filters is associated with a spatial area of theimage sensor.
 4. The method of claim 1, wherein each optical filter of agroup of optical filters is adjacent to another optical filter of thegroup of optical filters.
 5. The method of claim 1, wherein at least aportion of the optical filters are interference filters.
 6. The methodof claim 1, wherein the comparing a sampled output for each opticalsensor of the plurality of optical sensors with an expected outputcomprises: comparing the sampled output of at least some optical sensorsassociated with optical filters within one or more groups of opticalfilters with other optical sensors associated with optical filterswithin a same one or more groups of optical filters.
 7. The method ofclaim 6, further comprising: comparing the output of the at least someoptical associated with optical filters configured to pass light insubstantially the same wavelength range in different groups of theplurality of groups of optical filters; based on the comparing,determine an additional one or more calibration factors for one or moreoptical sensors of the plurality of optical sensors; and store theadditional one or more calibration factors in memory.
 8. The method ofclaim 1, wherein the image sensor is fabricated on an integratedcircuit, wherein the expected output corresponds to data stored inanother memory, wherein the another memory is located on the integratedcircuit.
 9. The method of claim 1, wherein the image sensor isfabricated on an integrated circuit, wherein the memory is located on asame integrated circuit with the image sensor.
 10. A spectral imagingsystem comprises: an array of optical sensors arranged on an integratedcircuit, the array of optical sensors having a respective top surface; aplurality of optical filters having a respective top surface and arespective bottom surface, wherein the bottom surface of the pluralityof optical filters is located proximal to the top surface of the arrayof optical sensors, wherein each optical sensor is associated with oneor more optical filters, wherein each optical filter is associated witha group of optical filters of a plurality of groups of optical filters,wherein each optical filter of a group of optical filters is configuredto pass light in a different wavelength range and wherein at least someoptical filters in different groups of the plurality of groups ofoptical filters are configured to pass light in substantially a samewavelength range; an interface; a local memory; and a processing moduleoperably coupled to the interface and the local memory, wherein theprocessing module functions to: sample an output from each opticalsensor of the array of optical sensors, wherein the output is based onthe array of optical sensors being illuminated with a light spectrumthat includes light of known wavelength and intensity; compare thesampled output for each optical sensor of the array of optical sensorswith an expected output; based on the comparing, generate a calibrationfactor for each of at least a subset of the array of optical sensors;and store the calibration factor for each of the at least a subset ofthe array of optical sensors in memory.
 11. The spectral imaging systemof claim 10, wherein the optical filters of each group of opticalfilters are further configured to provide a spectrum of wavelengths. 12.The spectral imaging system of claim 10, wherein each group of opticalfilters of the plurality of optical filters is associated with a spatialarea of the array of optical sensors.
 13. The spectral imaging system ofclaim 10, wherein each optical filter of a group of optical filters isadjacent to another optical filter of the group of optical filters. 14.The spectral imaging system of claim 10, wherein at least a portion ofthe optical filters are interference filters.
 15. The spectral imagingsystem of claim 10, wherein the processing module further functions to:compare the sampled output of at least some optical sensors associatedwith optical filters within one or more groups of optical filters withother optical sensors associated with optical filters within a same oneor more groups of optical filters.
 16. The spectral imaging system ofclaim 15, wherein the processing module further functions to: comparethe output of the at least some optical associated with optical filtersconfigured to pass light in substantially the same wavelength range indifferent groups of the plurality of groups of optical filters; based onthe comparison, determine an additional one or more calibration factorsfor one or more optical sensors of the plurality of optical sensors; andstore the additional one or more calibration factors in memory.
 17. Thespectral imaging system of claim 15, wherein the expected outputcorresponds to data stored in another memory, wherein the another memoryis fabricated on the integrated circuit.
 18. The spectral imaging systemof claim 15, wherein the memory is fabricated on the integrated circuitwith the image sensor.
 19. A method comprises: sampling an output foreach optical sensor of a plurality of optical sensors of an imagesensor, wherein each optical sensor is associated with one or moreoptical filters, wherein each optical filter is associated with a groupof optical filters of a plurality of groups of optical filters, whereineach optical filter of a group of optical filters is configured to passlight in a different wavelength range and wherein at least some opticalfilters in different groups of the plurality of groups of opticalfilters are configured to pass light in substantially a same wavelengthrange; and correcting the sampled output for each optical sensor of theplurality of optical sensors using a predetermined calibration factorstored in memory, wherein the predetermined calibration factor is basedon a comparison between a calibration sample output and a sample outputfor light of known wavelength and intensity.
 20. The method of claim 19,wherein the image sensor is fabricated on an integrated circuit, whereinthe memory is located on a same integrated circuit with the imagesensor.